Image synthesis is the final production stage of computer graphics image (CGI) related industry. The quality of images generated by computer graphics software (renderer) is crucial to cartoon animation film, TV program, and Visual effect (VFX) generation.
As illustrated on FIG. 1, ray tracing is the universal algorithm used to generate synthesized images. Rays are generated and shot from the view point. If a ray intersects an object in the scene, the color value at the intersection point is calculated, and the corresponding pixel value in image plane is set accordingly.
However, this simple ray tracing algorithm generates aliased image due to the lack of samples. Monte Carlo ray tracing algorithm is thus used to solve the aliasing problem. In a Monte Carlo ray tracing system, multiple samples (multiple rays) are generated and shot into the scene for each pixel, and the filtered color value is used as the pixel value.
Monte Carlo ray tracing algorithm is used by production quality render (Mental ray) to generate synthesized images in films and TV programs. In Monte Carlo ray tracing software, Monte Carlo integral is used to solve the rendering equation [see Kajiya, James T. (1986), “The rendering equation”, SIGGRAPH 1986] numerically (Equation 1).L(x→Θ)=Le(x→Θ)+∫Ωf r(x, ψΘ)Li(x→ψ)cos(N, ψ)dω  (1)
As illustrated on FIG. 2, the Monte Carlo integral converges to the true value using infinite samples, but with finite number of samples, variance always exists as noise in the synthesized image.
Improving the number of samples reduces the noise in a synthesized image, but with more computation cost. In ambient occlusion algorithm, shadow algorithm, and area lighting algorithm, the noise in synthesized images is reduced by increasing the number of samples per pixel.
There is lack of quality metric for synthesized images, especially for Monte Carlo ray traced images. The quality of a synthesized image is determined subjectively normally, and the number of samples per pixel used to generate “good quality” images is determined by user's experience, which is always conservatively estimated, with the penalty of costly and unnecessary computation. This situation impedes the cost of CGI's production time.
There are some research works targeting at the “perception based rendering”. In Mahesh Ramasubramanian, S. N. Pattanaik , Donald P. Greenberg, “A Perceptually Based Physical Error Metric for Realistic Image Synthesis”, SIGGRAPH 1999, human perception model is used to predict the quality metric, and this metric is used in recursive type global illumination rendering algorithms. However, the quality metric is only calculated from direct illumination rendering result, the quality metric for final rendered image is not considered in their work.
The rendering systems of the state of the art do not take noise artifact detection into consideration. The noise artifact detection and reduction are performed based on the experience of artists. The conservative estimation of samples per pixel always leads to rendering computations more than necessary.